Enable Text2text task on ipex #1054
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
Hi @echarlaix @IlyasMoutawwakil . Please review this PR, thanks! BTW, the failed tests are not related to my changes. |
| @@ -14,7 +14,7 @@ Optimum Intel can be used to load models from the [Hub](https://huggingface.co/m | |||
|
|
|||
| ## Loading | |||
|
|
|||
| You can load your model and apply IPEX optimizations (apply torch.compile for non-generation tasks). For supported architectures like LLaMA, BERT and ViT, further optimizations will be applied by patching the model to use custom operators. | |||
| You can load your model and apply IPEX optimizations (apply torch.compile except text-generation tasks). For supported architectures like LLaMA, BERT and ViT, further optimizations will be applied by patching the model to use custom operators. | |||
There was a problem hiding this comment.
I don't understand what you mean here, I see torch.compile being applied to text-generation task, why does it say "except text-generation tasks" here ?
There was a problem hiding this comment.
Actually, we didn't apply torch.compile in text-generation task which means IPEXModelForCausalLM doesn't have torch.compile in init. And generation tasks are also excluded in IPEXModel.init when calling torch.compile
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
|
Hi @IlyasMoutawwakil . I have fixed your comments, please take the 2nd round review. Thanks! |
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
|
Hi @IlyasMoutawwakil . It should be ready to merge, thanks! |
Add IPEXModelForSeq2SeqLM support, optimized by torch.compile. It forced
attn_implementation="static"to be compatible with torch.compile.